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ABSTRACT 



This paper reports on an assessment of the cognitive skills 
section of the "Test of Communication Knowledge." The test was designed by 
Dr. Rebecca B. Rubin to evaluate learning by undergraduate communication 
majors in the School of Communication Studies at Kent State University 
(Ohio) . In particular, the test is based on a set of communication skills 
identified as important by a panel of educators, employers, and policy makers 
and comes from a project designed to meet National Education Goal 5.5, which 
demands the identification of speaking, listening, writing, and critical 
thinking skills that should be possessed by college graduates (Jones, 1994) . 
The primary concern in this paper was the general reliability and validity of 
the cognitive skills section of the test. However, the item discrimination 
index was used to assess the ability of a test question to discriminate 
between those who did well on the exam from those who did not do well (i.e., 
between an expert group and a novice group) . Preliminary results suggest that 
some of the items in the cognitive skills section of the test could be 
improved to be more reliable and valid. (The cognitive skills section of the 
"Test of Communication Knowledge" (Revised) , the categorized skills college 
graduates should possess, and the results of qualitative and quantitative 
item analysis are appended.) Contains 17 references. (Author) 
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Abstract 

This paper reports on an assessment of the cognitive skills section of the "Test of 
Communication Knowledge." The test was designed by Dr. Rebecca B. Rubin to evaluate 
learning by undergraduate communication majors in the School of Communication Studies at 
Kent State University. In particular, the test is based on a set of communication skills identified 
as important by a panel of educators, employers, and policy makers and comes from a project 
designed to meet National Education Goal 5.5, which demands the identification of speaking, 
listening, writing, and critical thinking skills that should be possessed by college graduates 
(Jones, 1994). Our primary concern in this paper was the general reliability and validity of the 
cognitive skills section of the test. However, we used the item discrimination index to assess the 
ability of a test question to discriminate between those that did well on the exam from those that 
did not do well (i.e., between an expert group and a novice group). Preliminary results suggest 
that some of the items in the cognitive skills section of the test could be improved to be more 
reliable and valid. 
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Assessing Knowledge and Cognitive Skills 

Communication competence has been studied in terms of three components: knowledge, 
skill, and motivation (Spitzberg, 1983). The cognitive component can be conceptualized in terms 
of knowing, the skill component in terms of exhibiting, and the motivation component in terms 
of the incentive to engage in effective and appropriate communication acts. This review focuses 
on the knowledge, or cognitive, component of this triadic conceptualization. 

It seems that few other communication constructs have aroused so much attention and 
controversy in the literature over the years than communication competence. Perhaps this is 
because of its demand for applicability in educational settings at the end of the 1970s, when the 
state of Massachusetts implemented a basic skills requirement of its students. 

Whatever the reason, most researchers seem to agree that cognition plays an important 
role in the competence construct. Cognition may be separated into two structural processes: 
conceptual , or knowing that, and procedural , or knowing how (Anderson, 1976). These two 
structures account for a person's knowing why he or she does what he or she does. Cognitive 
skills encompass both conceptual and procedural knowledge. 

Greene (1984) posited the action assembly theory, which describes the relationship of 
conceptual to procedural knowledge in producing effective communication performances. He 
wrote that, 

procedural knowledge refers to stored action specifications, at a range of levels of 
abstraction, upon which an individual may draw in order to accomplish his or her desired 
ends. In short, the procedural store represents those things we have learned to do, and not 
to do, in order to act efficaciously, (p. 291) 

A rival explanation of cognitive processing is schema theory. Rumelhart (1980) and 
Thomdyke (1984) referred to schemas as knowledge structures that organize a person's memory 
for people and events. Within schemas operate cognitive scripts, which refer to schemas about 
particular activities. 
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Duran and Spitzberg (1995) noted that cognitively competent people tend to focus more 
on self-presentation and communication in general. Duran and Spitzberg conceptualized 
cognitive communication competence as having four functions: the anticipation of situational 
variables surrounding the communication act, the perception of the potential consequences 
regarding one's communication behavior, the immediate reflection of one's communication 
behavior, and the continued reflection of one's communication behavior. 

Duran and Spitzberg (1995) also factor analyzed the Cognitive Communication 
Competence Scale (CCCS). They found a temporal sequence of five dimensions of 
communication cognitions: planning cognitions, presence cognitions, modeling cognitions, 
reflection cognitions, and consequence cognitions. Planning cognitions refer to the mental 
rehearsal of conversational topics before interaction takes place. Presence cognitions monitor the 
alter-interactant during conversation. Modeling cognitions refer to the social awareness of what 
types of communication activities will occur in a situation. Reflection cognitions serve a heuristic 
function by serving as feedback and instructing the interactant how to improve a future 
performance. Consequence cognitions reflect a general awareness of the effects of one's 
communication performance. Therefore, a competent communicator should not only possess the 
knowledge and skill but should also know when and why certain skills should be activated. 

The cognitive skills section of the "Test of Communication Knowledge" consists of 50 
questions about 10 different communication scenarios. Students are asked to imagine themselves 
in the scenarios and then are asked five questions about each scenario (see Appendix A). This 
test was developed by Dr. Rebecca B. Rubin and is based on a set of communication skills 
identified as important by a panel of educators, employers, and policy makers (Jones, 1994). 

Specifically the set of communication skills comes from a project designed to meet 
National Education Goal 5.5, which demands the identification of speaking and listening, 
writing, and critical thinking skills that should be possessed by college graduates (Jones, 1994). 
Dr. Rubin grouped these speaking and listening skills according to subcategories reported in Daly 
(1992). These subcategories are informing, persuading, and relating. Each question in the 
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cognitive section of the "Test of Communication Knowledge" taps either speaking or listening in 
one of these three areas (see Appendix B for the skills and their categories). 

This project examined the knowledge and cognitive skills sections of the "Test of 
Communication Knowledge" in order to determine whether any items needed to be changed. The 
goal of this project was to revise the cognitive skills test and make it a more valid and reliable 
instrument. 

Method/Preliminary Results 

Sample 

The data for this study were collected from 244 undergraduates during the spring and 
summer semesters 1 994 at a large Midwestern university. 

Validity 

To assess discriminant validity we administered the test to two distinct learning groups. 
The sample of 244 undergraduates was comprised of 75 communication majors (expert group) 
and 169 non-majors (novice group). The competence instrument consists of two parts. Part one is 
a test of communication knowledge and uses a multiple choice format designed to assess if 
students have acquired knowledge about topics covered in communication classes. Part two, the 
cognitive skills test, presents various scenarios and asks students to make judgments about those 
scenarios. Each part has, and counts for, fifty points. This project only focused on part two of the 
"Test of Communication Knowledge." 

Test questions are scored as either correct or incorrect (scores may range from 0 to 50). 
Scores for communication majors ranged from 15 to 44, with a mean of 32.3, median of 33.9, 
and standard deviation of 6. 1 . Scores for non-majors ranged from 10 to 44, with a mean of 32.1, 
median of 33.5 and standard deviation of 6.3. We used a t-test to see if the experts and novices 
differed. The t-value was not significant (t(242) = 0.23, p > .05). 

To assess content validity we used an expert panel to determine the correct answers for 
each question. Members of the panel were 1 1 well-known professors in the communication field. 
If the experts disagreed on the correct answer for a particular question we reassessed the wording 
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of the question and its corresponding answers in order to make the correct answer the most 
obvious answer. We also considered the answers given by the subjects when revising the test. If a 
majority of students got a particular question wrong, or if there was a great deal of disagreement 
we examined the items in question. The goal was not to create questions that all students would 
correctly answer but rather to use this information as evidence of the possible existence of 
unclear questions or answer choices. For a complete summary of each item and changes see 
Appendix C. 

Another way we assessed content validity was to compare distribution of skills in the 
cognitive test with the distribution of skills from the original categorization by R. Rubin. We 
found that the speaking/persuading skills were under-represented in the test (see Appendix D). 
Reliability 

Overall reliability of the test administrations, measured with the KR-20 internal 
consistency statistic, was .77 for non-majors and .75 for majors. The reliability estimates for the 
speaking questions were .57 and .54 respectively. The estimates for listening items were .63 for 
non-majors and .56 for majors. 

The item discrimination index allows researchers to assess the ability of a test question to 
discriminate between those that did well on the exam from those that did not do well. The 
discrimination index compares the performance of the top scoring 27% to the lower 27% on each 
item. The index ranges from -1.00 to 1.00. A value of 1.00 indicates perfect discrimination 
between upper and lower students. A value of -1.00 indicates that the lower scoring students did 
better on an item than the upper students. A discrimination value of zero indicates no 
discrimination between the upper and lower groups and indicates that item may be too hard or 
too easy. A mean discrimination index can also be computed with values over .30 indicating 
good discrimination for the entire test. Two questions in the test yielded negative values on the 
discrimination index. These questions were subsequently reworded. All other test questions had 
positive index values ranging from .08 to .65. However, some questions that a significant portion 
of both majors and non-majors answered incorrectly were reworded to help increase validity and 
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reliability. The mean discrimination indices for the two administrations of the cognitive skills test 
were .29 for non-majors and .28 for majors, possibly suggesting the need for further refinement 
of the original test. 

Implications/Future Directions 

This project has focused on the "Test of Communication Knowledge" which has been 
administered to the majors and pre-majors of communication studies at Kent State University 
every other year for the past six years. In particular, the goal of the project was to examine the 
items in part two of the knowledge test in order to (a) conduct an item analysis to determine 
which items, if any, needed to be reworded and/or deleted from the test; (b) examine the skills 
being assessed to determine if additional skills needed to be included; and (c) develop a more 
valid and reliable instrument designed to assess the necessary skills that all Kent State graduates 
should possess. Since this project was conducted, the revised cognitive skills section of the “Test 
of Communication Knowledge” has been administered to both majors and pre-majors. The 
authors plan on continuing this study by comparing the 1994 data with the current data to see if 
the validity, reliability, and the item discrimination indexes have changed. 

Although Backlund (1983) has argued that "teacher-constructed" tests and procedures 
designed to assess competence lack reliability and validity, the efforts involved in this project 
(and future projects) have been geared toward developing a standardized test that is both valid 
and reliable. Indeed, because many instruments have been designed to assess competence (e.g., 
CAS, CCS, CCAI), it appears that scales are valid only for the use to which they are put. In other 
words, the "Test of Communication Knowledge" is designed to assess the knowledge and skills 
that have been taught to the undergraduate students at Kent State University. 

We recommend that future administrations of this test (i.e., cognitive skills section) be 
done separately from the knowledge portion because of the sheer number of items. In previous 
administrations, the cognitive skills section was given to all subjects after they completed the 
knowledge section. This may have led to fatigue as the entire test contained a total of 100 items. 
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A second related reason for separate administration is that the cognitive items are in the form of 
scenarios, which require deeper processing and concentration on the part of the respondents. 

To make the test more representative of the skills identified by Jones (1994) and 
classified by Rubin, we recommend adding two scenarios that focus on speaking/persuading. The 
addition of 10 speaking/persuading items may give the test a similar distribution to that of the 
original list of skills. 

Because programs at schools and universities differ, it seems appropriate that the 
instruments designed to assess competence be structured around the program itself. Part one of 
the knowledge test assesses what Royer, Cicero, and Carlo (1993) have identified as "knowledge 
acquisition" which uses true-false, matching, and multiple choice items to assess if students have 
acquired knowledge. Part two of the knowledge test assess what Royer et al. (1993) identified as 
"depth of problem representation" which asks students to make judgments about problems, 
situations, or scenarios. 

Rubin (1983) and others (e.g., Spitzberg, 1983) have argued that students should be 
evaluated in the areas of knowledge, skill, and motivation so that programs may focus on the 
deficient areas. We believe the "Test of Communication Knowledge" assesses both the 
knowledge (part 1) and skill (part 2) of the student; however, future projects may want to include 
a part designed to assess the motivation of the students at Kent State University. Indeed, scholars 
have investigated student motivation to study and its correlates (e.g., Christophel, 1990; 
Christophel & Gorham, 1995; Frymier, 1993a, 1993b; Frymier & Shulman, 1995). For example, 
Frymier and Shulman (1995) found that relevance is associated with a student’s state motivation 
to study. 

In addition, Christophel and Gorham (1995) found a causal relationship between teacher 
immediacy and state motivation. Furthermore, they found that students perceive motivation as a 
personally-owned state and demotivation as a teacher-owned state. Therefore, it is apparent that 
we (as scholars, teachers, and academicians) need to learn more about student motivation if we 
want to help student’s motivation and knowledge and skill. Because the ideal "communicatively 
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competent" student should also exhibit listening and speaking competence, researchers may 
consider developing sections designed to assess those skills as well. 

The ultimate goal of any educational institution is to provide an effective undergraduate 
(or graduate) education that helps students become better thinkers, communicators, and decision 
makers in the "real world." By keeping this goal in mind, programs should focus on assessing the 
knowledge and skill taught in the classroom. Furthermore, programs should focus on assessing 
the skills needed for today's graduate to not only survive in the "real world," but to prosper and 
excel in it as well. The competent student should possess the knowledge, skill, and motivation to 
behave in an appropriate manner. Parks (1994) stated it best when he wrote "to be competent 
therefore we must not only 'know' and 'know how,' we must 'do' and 'know what we did'" (p. 

591). 
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Appendix A - Cognitive Skills Section of the “Test of Communication Knowledge” (Revised) 

Each of the following questions focuses on a scenario involving you in a communication 
situation. Read through the scenario first and then answer the questions. Because the scenarios 
change, it is best that you answer all the questions for the scenario before proceeding on to the 
next one. If more than one answer seems correct, choose the one that is most correct. 



Scenario A — Class Project 

You're working on a class project with four other students. People have looked to you to 
coordinate the group activities. You have a month to finish the project, so you feel it's about time 
to get started. Your group agrees to meet every Monday evening from 7 p.m. to 1 1 p.m. 

71 . One group member arrived at the first meeting late and left early. She rarely established 
eye contact with other group members and sat off in a comer most of the evening. She 
said she was highly involved in the project and volunteered to bring important materials 
to the next meeting. Your assessment is: 

a. she's involved in the project and most probably will be dependable 

b. she may not be dependable, even though she says she is 

c. she's definitely not going to carry her weight on the project 

d. Monday wasn't a good night for the group meetings 

72. As group leader, you observe this group member's first evening's behavior. You should: 

a. ask her if there is a problem with the meeting time or project 

b. wait for a few weeks and then say something to her if the same behavior 
continues 

c. ask her to sit with the rest of the group and to arrive on time 

d. not pay attention to the nonverbal behavior; she said she was involved and 
interested so you should believe her 

73. The next time you meet and she is again quiet, you should: 

a. let her alone; she'll speak when she has something to say 

b. ask her a question about her already-researched content area 

c. ask her a question about a new area your group has not yet discussed 

d. ask her to stand up before the group and summarize the last meeting 

74. Your job as group leader is to: 

a. make sure everyone agrees 

b. start and end meetings on time 

c. let the discussions go where they may 

d. keep group discussions relevant and focused 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

75. By the end of this evening's meeting, your group must finalize its plans. The best way for 

you to help the group build consensus is for you to: 

a. use Robert's Rules of Order 

b. summarize points of agreement and dissect issues yet unresolved 

c. have the four group members debate, in teams, the issues 

d. take a vote for each new decision 



Scenario B — Search Committee 

You've been selected the student member of a committee that is conducting a search for a new 
faculty member. The finalists have been invited to visit campus and the first one is to arrive 
today. 

76. When you first meet the candidate you introduce yourself by saying: 

a. Hi. What makes you want to teach as State U? 

b. Hi, I'm . I'm the student representative to the search committee. 

c. Hi, I'm . I'm supposed to show you around campus. 

d. Hi, I'm a senior here. Ask me anything. 

77. You would like to say something nice about the candidate's qualifications. The most 
graceful way to say this is: 

a. You sure have done a lot of research! 

b. I've admired your career. 

c. Your research record is impressive! How did you get started? 

d. Why would anyone want to do so much writing? 

78. To give the applicant an idea of what students are like on your campus, you'd best bring 
up topics such as: 

a. summer jobs, graduation, football 

b. salary, religion, family history 

c. student background, future aspirations, internships 

d. pets, family members, recent holiday visits 

79. One of your jobs is to lead discussions between the candidate and student groups. During 
one such discussion, silence erupts for a bit too long. Your best course of action is to: 

a. end the meeting 

b. ask the candidate a question 

c. offer your own opinions 

d. ask a student a question 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

80. When you last see the candidate, you say, "It was a pleasure to meet you... 

a. Have a safe trip home 

b. You're better than the last applicant 

c. Good luck in the future 

d. I hope you get the job 



Scenario C — Peer Advising 

As part of a peer advising program, you've volunteered to counsel new Freshmen in the major. 
You're to meet with a group of 20 new students tomorrow evening to help them prepare for 
registration the next day. Your job tomorrow is to give directions on how to register and on what 
courses would be best for them to take. 

81 . What is the best organizational plan for this talk? 

a. List on the blackboard the directions for registering, and identify required 
courses. 

b. Have the students introduce themselves to one another, have them read about 
the courses in the catalog, tell them about your first registration, highlight all 
the things you did wrong the first time. 

c. Introduce yourself and the major, list and describe the courses, give directions 
for registering in order, summarize the important points. 

d. Introduce the faculty in the department, show the students around campus, give 
them directions to the classroom buildings, point out how to register in the 
timetable, highlight the important classes. 

82. Your student group is most concerned about how to register via telephone. This means 
that your talk: 

a. should concentrate on call-in registration 

b. should cover how to register via telephone, but should also explain 
departmental courses and faculty 

c. should make the Freshmen feel at home 

d. should cover how to register via telephone last, right before they leave 

83. If you had the time and resources, the best way to present this information would be via: 

a. computer terminal 

b. slide show 

c. written handout 

d. overhead projector screen 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

84. When preparing your directions on how to register, a good rule to follow is: 

a. give a list of every single step in chronological order 

b. list the major steps and break each one down in your explanation 

c. give a general picture of what one goes through 

d. let students learn by doing; don't give directions 

85. A student asks a question about how to get an advisor. Your answer should: 

a. be concise, to the point 

b. be thorough, explained at length 

c. be funny, friendly 

d. lead the student to the answer, but not answer it directly 



Scenario D — Informal Meeting 

You meet with several students regularly during a 30-minute break between classes. Mostly you 
all talk about your classes, professors, and people you all know. 

86. Chris, Lee, and Pat are sitting around drinking coffee when you arrive. They're very 
involved in conversation about the exam that was just returned in their class. You, 
however, are more concerned about the exam you're all scheduled to take that is coming 
up next hour. It is appropriate for you to interrupt their conversation: 

a. if you have information that would be helpful to their taking the next exam. 

b. because you always interrupt each other; topics change minute by minute. 

c. as soon as you arrive, because you haven't seen them for a week and you have 
the right to change the topic. 

d. it is never appropriate to interrupt them. 

87. The best way to empathize with them is to: 

a. tell them about the last exam you had 

b. talk about the exam that is coming up 

c. imagine how they must feel and describe these feelings 

d. change the subject so they won't worry about the next exam 

88. Robin stops by the group and tells Chris, Lee, and Pat that they must not have studied for 
the exam because last semester the same exam was so easy almost everyone got A's and 
B's. Your response to Robin should be: 

a. to allow these differing viewpoints to emerge 

b. to explain that circumstances might be different this year 

c. to argue that three people could not be wrong whereas one could 

d. to avoid Robin in the future 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

*89. You want to show Robin you understand these views (see above question). A good 
paraphrase would be: 

a. So you think the teacher's an easy grader? 

b. So you felt the exam last semester was easy because you and everyone else 
studied and did well? 

c. So you think Chris, Lee, and Pat are poor students because they goof off? 

d. So you think Chris, Lee, and Pat should see you before the next exam since the 
teacher uses the same exam each semester? 

90. How can you best accomplish the multiple goals of keeping Robin, Chris, Lee, and Pat 
friends and persuading all of them to study harder next semester? 

a. Tell Chris, Lee, and Pat that they're not studying hard enough 

b. Tell Robin the exam must have changed 

c. Focus the discussion on how Robin can help with the next exam 

d. Change the topic so they won't get into an argument 



Scenario E -- Scholarship Plan 

You've developed a new plan for raising scholarship money and the department chair has asked 
you to present your plan to the faculty (about 20 in number), who have the power to 
approve/implement the plan or to disapprove it. Your presentation is at the end of the week and 
during the time you have available you much consider the best method of structuring your 
arguments. 

91 . Instead of the 20 faculty you expected, only four show up for your presentation. 

a. This makes no difference. You proceed as planned. 

b. You proceed as planned, but sit around the table instead of standing. 

c. You throw away everything you planned and discuss your plan 
extemporaneously. 

d. You use the general structure you planned to use and the evidence, but use a 
less formal tone. 

92. The best way to begin the presentation would be: 

a. to use an interesting attention-getter 

b. to start off with the first main point 

c. to begin with your conclusions, what you want them to do 

d. to disguise the purpose of your speech at first 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

93. You can establish your credibility by: 

a. hinting at your GPA in the major 

b. citing sources, developing arguments, using supporting material 

c. meeting everyone prior to the presentation 

d. using a laid-back, non-aggressive style 

*94. During your talk you watch the faculty to see how they are reacting. You would most 
probably think they understand what you're saying if they: 

a. begin talking with each other 

b. look around the room 

c. use backchanneling responses such as "uh-huh" or "yes" 

d. glance at each other 

*95. You would most probably think they disagree with you if they: 

a. nod their heads 

b. smile at you 

c. shake their heads 

d. take notes 



Scenario F — Class Changes 

One of your professors has changed some vital aspects of the syllabus and has asked the teaching 
assistant (TA) to call each student to inform him or her of the changes. You have just answered 
the phone. The TA says, "Professor Jones has asked me to call all students to let them know that 
he's been called out of town for the next two weeks and classes will resume the week after that. 
You're supposed to continue working on your projects and continue reading the text. Everything 
will be shifted around once he gets back, so if you have the readings all done you'll be able to 
adjust to the new schedule for the remainder of the class." 

96. The main point in this message is: 

a. Professor Jones is out of town and class will be adjusted once he returns 

b. Work on projects and read the textbook for next two weeks while class is 
canceled 

c. Projects and readings are more important than class meetings 

d. Projects will be due two week later than originally scheduled 




* 

1C 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

97. During the explanation of changes, you're not quite sure whether the TA said that the date 
for the next exam had been changed or if it is as originally scheduled. It's best to: 

a. Ask the TA if the date for the next exam has been changed. 

b. Ask a friend in class if he/she knows about the next exam. 

c. Not ask and wait until it is discussed in class. 

d. Plan on taking it on the date given in the syllabus. 

98. The TA has a very distinct accent and you find that you must listen very closely to what 
the TA is saying. Your best response to this is: 

a. Complaining to the department about the TA's accent 

b. Asking for a section shift to someone you can understand better 

c. Giving the TA a low evaluation at the end of the semester 

d. Listening carefully to understand what is being said 

99. Your TA comes from a culture where words have very precise meanings. How does this 
influence your phone conversation? 

a. It means you should place your own interpretation on what is said 

b. It means you should take everything said literally, at face value 

c. It means you should call Professor Jones when he returns to work 

d. Meanings are within people; your TA shouldn't try to change this culture 

* 1 00. Your roommates are loudly celebrating Friday afternoon when the phone call comes in. 
You should... 

a. reduce the noise level before the caller begins to speak 

b. ask the caller to call back on Monday 

c. hear as much as you can and then check with a friend to fill in 
anything missed 

d. ask the caller to speak louder and slower so you can hear 



Scenario G — Campus Speaker 

One of your classmates feels that a radical religious figure who has planned to visit campus 
should not be allowed to speak because of the hatred the speeches generate. Your classmate has 
asked you sign a petition banning this speaker from campus. 

101 . You wonder if your classmate has considered the first amendment rights of speakers. The 
best way to phrase your question is: 

a. So why don't you think he should get a chance to speak? 

b. Your solution sounds pretty drastic, doesn't it? 

c. Shouldn't all people have the right to express their ideas? 

d. Haven't you heard of the first amendment, Bozo? 



JL o 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

1 02. Your classmate gets into a lengthy discussion on the intent of the first amendment, citing 
the fact that it doesn't hold up in all situations (e.g., shouting "fire" in a crowded place). 
He tells you that this situation is just like a crowded theatre, in that what he says will 
incite a riot and people will be hurt, so it's O.K. to prevent harm to people by prohibiting 
the talk. Your classmate is basing his position on: 

a. fact, because it's a fact that this speech would cause harm to others 

b. fact, because it's based on a law, which overrides an amendment 

c. opinion, because it's his interpretation of legal statutes 

d. opinion, because anything he says is his opinion 

103. Your classmate's assumptions and conclusions, identified in #32, could be labeled: 

a. completely logical 

b. appeal to tradition 

c. bandwagon effect 

d. leap of inference 

*104. Your classmate happens to be a member of a religious group different than yours. You 
wonder if you should believe your classmate. You reason: 

a. both the message and the speaker must be credible to be believed 

b. as long as the speaker is credible, the message can be weak in logic 

c. as long as the message makes sense, the speaker's credibility can be in doubt 

d. your first hunch is the best, so both the speaker and message can be low in 
believability 

*105. You attend the speech and find you do not agree with the speaker on anything. You can 
best spend you time there... 

a. daydreaming about something else 

b. heckling the speaker 

c. thinking about how to use the information later 

d. reading a book 



Scenario H — Retiring Professor 

One of your favorite professors is retiring and several students, yourself included, have been 
asked to say a few words at the annual awards banquet. 

106. The best format for your remarks would be: 

a. an informative talk about your professor's career 

b. an entertaining toast to your professor's career 

c. an obituary or eulogy about your professor 

d. a persuasive appeal to faculty to hire someone as good or better 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

1 07. The best length for your remarks would be: 

a. 7-10 minutes 

b. 5-8 minutes 

c. 11-15 minutes 

d. 2-3 minutes 

1 08. You've written your remarks out (because the department is putting together a scrapbook 
containing all tributes), but when you arrive you see that there is no podium on which to 
rest your pages. Your best course of action is to: 

a. contribute the written remarks, but not deliver them in public 

b. read them as originally planned, holding the pages in your hand 

c. memorize the main points and present the remarks extemporaneously 

d. write the main points on your palm and refer to it when necessary 

1 09. The best way to refer to her during your talk is to use which of the following titles? 

a. Miss Barbara Brown 

b. Barb Brown 

c. Ms. Barbara Brown 

d. Professor Barbara Brown 

1 1 0. Which of the following phrases would be the most flattering tribute? 

a. She's been a role model to many women. 

b. She's been a mom to us all. 

c. She's a grand old girl. 

d. She's a typical female professor 



Scenario I — Advisor Appointment 

Every year your advisor schedules an appointment with you to talk about your progress in the 
major. This is your opportunity to give feedback to the faculty, and to receive feedback from 
your advisor on your progress. 

111. Your advisor appraises your record and you agree with everything she is saying. To let 
her know this you should: 

a. listen quietly and not say a word 

b. interrupt her with comments like "I agree," " You're right," etc. 

c. respond with "Uh-huh" and "yes" at the end of each point that you agree with 

d. After each point, paraphrase fully what she has just said 
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Appendix A - Cognitive Skills Section of the "Test of Communication Knowledge" (Revised) 
(cont.). 

112. You feel that there are too many required courses for the major. When she asks you how 
you feel about this, you reply: 

a. that you feel that there are too many requirements for the major 

b. that things are O.K. as they now stand 

c. that you don't really have an opinion about it 

d. that you think the university should allow more substitutions 

1 13. Your advisor compliments you on your grade point average. You best respond: 

a. by smiling 

b. by thanking her for noticing 

c. by stating that it isn't as high as you'd like 

d. by pointing out how so many others have higher GPA's 

1 14. Your advisor asks how you would change the major if you had the opportunity. The most 
appropriate feedback for you to give would 

a. describe problems students have had 

b. propose solutions to these problems 

c. focus on what works well with the major 

d. be to say nothing 

*115. During your last meeting with your advisor before you graduate, your goal should be: 

a. to get a good letter of recommendation 

b. to tell her about all the problems you've had over the years 

c. to summarize your college experience and identify possible meetings in the 
future 

d. to tell her about your plans for the summer 



NOTE: TURN OVER YOUR ANSWER SHEET FOR THE LAST FIVE QUESTIONS 
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Appendix A - Cognitive Skills Section of "Test of Communication Knowledge" (Revised) 

(cont.). 

Scenario J — Changing Major 

You're considering changing your major. A friend has invited you along to a student group in the 
major you're considering so that you can find out more about the major and students' goals. 

* 1 1 6. The group has been discussing the importance of attending graduate school to ensure 
career advancement. Pat thinks graduate school is important especially if you want to 
teach, Chris believes that graduate school can provide some important networking 
opportunities for that first job, and Lee believes that graduate school helps students polish 
their skills as well as learn new information. These three people: 

a. believe that graduate school is essential 

b. do not agree on the importance of a graduate education 

c. believe that graduate school is only for those who want to teach 

d. agree that graduate school would be useful 

1 1 7. The goal of the discussion described in the above question is to: 

a. Explore possible uses of a graduate education in the major 

b. Debate the pros and cons of attending graduate school 

c. Convince all majors to apply to graduate school 

d. Encourage majors to get jobs upon graduation instead of continuing in school 

118. You have a cousin who couldn't get a job, even with a master's degree in this field. 
Considering your role in this group, it is best for you to: 

a. Argue that a graduate education guarantees nothing 

b. Explain at length what your cousin went through to find a job 

c. Ask about employment opportunities in the field 

d. Convince the group to focus on getting a job instead of more schooling 

1 19. The group breaks into a rousing discussion about how to raise funds for social gatherings. 
You have a great idea... 

a. but say nothing, because you want to give the idea to your own group 

b. but say nothing, because you're not a member of the group 

c. and you tell them your idea, because they need help 

d. and you tell them your idea, because you want to get credit for the idea 
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Appendix A - Cognitive Skills Section of "Test of Communication Knowledge" (Revised) 

(cont.). 

120. Imagine you want to join this group, but you have a class the night that they meet. Which 
of the following tactics would most effectively solve this problem? 

a. cite statistics about meeting attendance for different nights of the week 

b. bring in letters from faculty who agree with the change of day 

c. use an emotional appeal that you really need this group and they owe it to you 
to change 

d. negotiate by proposing a different night, evaluate the response, and amend the 
proposal to one that is more acceptable (e.g., time change on same day) 



*Note: Those questions indicated with an asterisk (*) are the items which we reworded either the 
question and/or the possible answers to increase reliability, validity, and the item discrimination 
index (For your convenience, those questions include 89, 94, 95, 100, 104, 105, 115, and 1 16). 



WHEN FINISHED, INSERT YOUR ANSWER SHEET IN THIS QUESTION BOOKLET 
AND RETURN IT TO YOUR PROFESSOR OR TEST PROCTOR. THANK YOU FOR 
COOPERATING! 
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Appendix B - Categorized Skills College Graduates Should Possess 

SPEAKING - Persuading 

1 . Communicate ethically (e.g., present information accurately, reveal purpose, avoid deception). 

2. State ideas clearly. 

3. Establish a relationship among ideas within a variety of oral communication contexts. 

4. Choose topic & message with which they are comfortable & knowledgeable. 

5. Identify their communication goals. 

6. Accomplish their communication goals. 

7. Structure a message for effectiveness with an introduction, main points, useful transitions, & a 
conclusion. 

8. Develop & present an interesting & attention-getting introduction in a speech. 

9. Use summary statements in appropriate contexts and situations. 

1 0. Adapt communication styles to the context(s) and situation(s). 

1 1 . Prepare a message reflecting the available information about the audience. 

12. Adapt to changes in audience characteristics (e.g., size, interests, concerns, heterogeneity). 

13. Choose & narrow a topic according to the needs of the audience & according to the occasion. 

14. Identify facts, issues, & problems relevant to the task at hand. 

15. Provide appropriate supporting material based on audience, occasion, & purpose. 

16. Demonstrate credibility through choosing appropriate material, citing sources, developing main ideas. 

1 7. State intentions & purposes when appropriate. 

1 8. Use motivational appeals that build on values, expectations & needs of the audience. 

19. Research effectively information required for message preparation (e.g., seek information in books & 
periodicals, identify & question appropriate authorities). 

20. Incorporate information from a variety of sources to support message. 

21 . Support message by incorporating statements of others (quoting correctly or paraphrasing objectively) 
into their own statements. 

22. Identify & use appropriate statistics to support the message. 

23. Recognize & be able to use basic reasoning (e.g., draw specific conclusion from general information, 
extrapolate general conclusion from specific information). 

24. Support arguments with relevant & adequate evidence. 

25. Incorporate language that captures and maintains audience interest in message. 

26. Use humor when appropriate (e.g., stories, anecdotes). 

27. Develop messages that influence attitudes, beliefs & actions. 

28. Convey enthusiasm for topic through delivery (e.g., vary pitch, volume, & tone; use appropriate 
stance, posture, eye contact to develop rapport with audience). 

29. Negotiate effectively (e.g., as a potential employer proposes a salary, evaluate response, amend 
proposal as necessary). 

30. Motivate others to participate and work effectively as a team. 

31. Use visual & other aids to support ideas, motivate, & persuade others. 
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Appendix B - Categorized Skills College Graduates Should Possess (cont.) 

SPEAKING - Informing 

1 . Recognize when it's appropriate to communicate. 

2. Communicate candidly. 

3. Be aware of language indicating bias on gender, age, ethnic, or sexual/affectional orientation (e.g., 
avoid sexist, racist, or homophobic language). 

4. Select the most appropriate and effective medium for communicating. 

5. Choose appropriate & effective organizing methods for messages; organized so that it aids listeners in 
understanding material. 

6. Outline the key points & sub-points of their spoken message. 

7. Understand their roles in a variety of settings (main speaker vs. speaker who introduces the main 
speaker). 

8. Prepare a message reflecting the available information at the context(s) and situation(s) in which oral 
communication occurs. 

9. Prepare a message reflecting the available information on the physical setting. 

10. Adapt to the physical setting (e.g., lighting, size of room, equipment). 

1 1 . Demonstrate competence & comfort with information. 

12. Compose & deliver an informative speech. 

13. Deliver an impromptu or extemporaneous talk about topics with which the speaker is familiar. 

14. Answer questions concisely. 

15. Give concise & accurate directions. 

16. Conduct & participate in an interview. 

17. Match people, interests, & tasks when coordinating a group project. 

18. Use pronunciation, grammar, & articulation appropriate to the designated audience. 

19. Use appropriate nonverbal behaviors for the message. 

20. Use appropriate vocal behaviors for the message. 

21. Speak publicly or in conversational setting without displaying extreme anxiety or nervousness. 



SPEAKING - Relating 

1 . Compose & deliver an entertainment-oriented speech when appropriate (e.g., an after dinner speech, a 
toast). 

2. Describe or express feelings to others when appropriate. 

3. Identify & resolve misunderstandings. 

4. Identify & adapt to the perceived needs & desires of other communicators. 

5. Understand status & relationships between communicators (e.g., knowing when to use formal titles 
such as "doctor" or "your honor"). 

6. Make effective decisions during communication situations. 

7. Recognize time constraints of a communication situation & know how to operate within them. 

8. Adjust to factors that might inhibit effective communication (e.g., move away from crowds when 
conducting a serious conversation). 

9. Understand the influence of culture on language. 



Assessing Knowledge 26 



Appendix B - Categorized Skills College Graduates Should Possess (cont.) 

10. Recognize when it is inappropriate to schedule or participate in a meeting. 

1 1. Build & maintain relationships with superiors, peers, & subordinates. 

12. Break off relationships when appropriate. 

13. Effectively assert themselves when appropriate. 

14. Avoid conflict when appropriate (e.g., focus discussion on issues, avoid personal attacks, clarify 
position & objectives). 

15. Disclose appropriate information about themselves when relevant. 

16. Motivate others to disclose information when appropriate. 

1 7. Pursue multiple communication goals effectively (e.g., persuade while maintaining friendly relations). 

1 8. Begin & end a conversation in an appropriate & socially acceptable manner. 

1 9. Introduce new topics when appropriate. 

20. Have confidence to approach & engage in conversation with new people in new settings. 

21. Give bad news to others (e.g., tell boss when a business deal falls through, tell a co-worker when 
he/she has been fired). 

22. Give compliments gracefully. 

23. Work on collaborative projects as a team. 

24. Understand & complement different methods of building consensus. 

25. Foster a sense of community between group members. 

26. Generate conversation & discussion from quiet members of a group. 

27. Manage troublesome members in a group. 

28. Confront others effectively in appropriate contexts. 

29. Set & manage realistic agendas. 

30. Lead meetings effectively. 

3 1 . Keep group discussions relevant & focused. 



LISTENING - Persuading 

1 . Ask questions effectively. 

2. Demonstrate attentiveness through nonverbal & verbal behaviors (e.g., nod, respond with facial 
expressions, vocalize agreement by saying yes or uh-huh). 

3. Be open-minded at another's point of view. 

4. Focus without fear on speaker & message (e.g., eliminate or reduce barriers to effective listening 
caused by communication anxiety). 

5. Identify discrepancies between the speaker's verbal and nonverbal message. 

6. Recognize nonverbal behaviors (e.g., eye contact, gestures, facial expressions, stance, physical 
proximity). 

7. Respond appropriately to nonverbal behavior. 

8. Listen with an open mind. 

9. Distinguish main points from supporting detail. 

10. Distinguish facts from opinions. 

1 1 . Evaluate ways in which speaker's ideas might be applied in new or different situations. 
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Appendix B - Categorized Skills College Graduates Should Possess (cont.) 

12. Analyze assumptions, evidence, & conclusions of an argument (i.e., recognize fallacies, leaps of 
inference). 

13. Evaluate evidence on the basis of relevance, appropriateness, & adequacy. 

14. Detect & evaluate bias & prejudice. 

1 5. Evaluate speeches & messages on the basis of the credibility of both the speaker & message. 

16. Evaluate speeches & messages during the delivery of the message (e.g., organizational strategies, 
stylistic elements, relevance & adequacy of supporting material, argumentation). 

17. Know how to determine their listening goal & to match that objective to the speaker's goal. 

18. Recognize sarcasm & irony. 

19. Recognize when a speaker's credibility is questionable. 

LISTENING - Relating 

1 . Recognize when it is inappropriate to speak. 

2. Recognize when another does not understand their message. 

3. Reduce barriers or interference that may inhibit the communication process (e.g., internal or external 
noise). 

4. Understand & value differences in communication styles (e.g., speech that is difficult to understand 
due to a handicap, culture, or accent of the speaker). 

5. Allow others to express difficult views. 

6. Cope with negative feedback. 

7. Convey empathy when communicating. 

8. Give feedback appropriately (e.g., offer constructive criticism). 

9. Respond to feedback. 

1 0. Paraphrase or restate what speaker has said to confirm his/her meaning. 

1 1 . Sustain topics & offer follow-up comments when appropriate. 

12. Interrupt effectively when appropriate. 

13. Receive compliments gracefully. 

14. Allow others to take credit for cooperative efforts when appropriate. 

1 5. Listen empathetically to help speakers clarify their thoughts & feelings. 



LISTENING - Informing 

1 . Listen carefully to questions & comments from other communicators. 

2. Ask questions when they do not understand another's message. 

3. Identify points of agreement between communicators (group). 

4. Identify common purpose & interests of a group. 

5. Recognize their role in a small group (e.g., recognize when they are acting as a leader, facilitator, 
disrupter). 

6. Cope with unanticipated changes in the setting in which communication takes place (e.g., the room is 
smaller than expected). 



Assessing Knowledge 28 



Appendix B - Categorized Skills College Graduates Should Possess (cont.) 

7. Listen attentively. 

8. Identify important points when given oral instructions. 

9. Listen carefully to speakers with strongly accented or impaired speech. 

10. Reflect on what is being said. 

1 1 . Recognize speaker's purpose or goal. 




21 



Appendix C - Result of Qualitative and Quantitative Item Analysis. 
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Appendix C - Result of Qualitative and Quantitative Item Analysis (cont.). 
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Majors (51.5%) and Non-Majors (56%) answered A 
Majors (32%) and Non-Majors (28%) answered C 



Appendix C - Result of Qualitative and Quantitative Item Analysis (cont.). 
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Majors (8 1 .7%) and Non-Majors (96%) answered A 



Appendix C - Result of Qualitative and Quantitative Item Analysis (cont.). 
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Appendix C - Result of Qualitative and Quantitative Item Analysis (cont.)- 
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Appendix C - Result of Qualitative and Quantitative Item Analysis (cont.). 
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Appendix D - Skill content/distribution of test vs. skill list. 
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